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DETAILED ACTION 

Information Disclosure Statement 

1 . The references listed on the Information Disclosure Statement (IDS) 
submitted on 01/24/05 and 05/04/05 have been considered by the examiner 
(see attached PTO/SB/08). 

Claim Rejections - 35 USC § 103 

2. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described 
as set forth in section 1 02 of this title, if the differences between the subject matter sought to 
be patented and the prior art are such that the subject matter as a whole would have been 
obvious at the time the invention was made to a person having ordinary skill in the art to which 
said subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 

3. Claims 9-14, 16-18, 20-25, 27-28, 30, and 32 are rejected under 35 U.S.C. 
103(a) as being unpatentable over Chen (US Patent 6,556,704) in view of 
Tuomi (US Patent 7,061 ,507). 

Regarding claim 9, Chen discloses a method for adjusting image capture 
settings for an image capture device comprising the step of: 

- identifying a scene (i.e. searching for qualified feature points; col. 3, 
lines 58 - 67); 

- capturing an image of the scene (i.e. cameras are arranged to capture 
images; see figure 1 and col. 6 lines 40 - 57); 
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- and generating a depth mask of the scene from data defining the image of the 
scene (i.e. a foreground depth mask is generated using the depth map 
containing the foreground feature points; col. 4, lines 1 - 23). 
Chen does not explicitly teach the step of adjusting pixel values corresponding to 
objects within any one or both of a foreground region and a background region of the 
captured image. 

In same field of endeavor (depth mask), Tuomi teaches a method for generating' 
a depth mask which include the step of adjusting pixel values (e.g. Z-value) 
corresponding to objects within any one or both of a foreground region and a 
background region of the captured image (see figure 21, col. 12, line 60 to col. 13, 
line 36). Therefore, it would have been obvious to one of ordinary skill in the art at 
the time the invention was made to modify Chen by adjusting pixel values with 
relating to the depth value or a Z-value. The depth information in the Z-buffer 2606 
is primarily used during the rendering process to determine which pixel is in the 
foreground (see figures 21 and 26, and col. 16, lines 48 - 57). The motivation to do 
so would have benefit of reducing the amount of memory required for each pixel 
stored in a frame buffer because only the color values for the antialiasing mask 
operation need to be stored, thus reducing the size of the frame buffer (see col. 10, 
lines 57-61 of Tuomi). 

Regarding claim 1 7, this claim differs from claim9 only in that claim 9 is method 
where claim 17 is apparatus. Tuomi clearly teaches "the bit value of the depth mask" 
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as recited in claim 17 (see col. 13 lines 7-36). Thus it is analyzed as previously 
discussed with reject to claim 9 above. 

Regarding claim 25, this claim differs from claim 17 only in that the limitations "a 
computing device" and "a display device" are additionally recited. Chen teaches a 
computing device (1 1 2) and a display (114) (see figure 1 2 and col. 5, lines 56-67). 

Regarding claim 10, Chen teaches segmenting the foreground (e.g. 63) and 
background (e.g. 62) of the scene as broad claimed language (see col. 2, lines 50- 
60). 

Regarding claims 1 1 and 32, Chen teaches the data defining the image of the 
scene includes pixel data where each pixel is tagged with distance information (see 
col. 11, lines 16-31). 

Regarding claim 12, Tuomi teaches the method operation of independently 
adjusting pixel values associated with the foreground region from pixel value 
associated with background region (see figure 26 and col. 15, line 38 to col. 16, line 
56). 

Regarding claim 13, Chen teaches an image capture device comprising a digital 
camera (col. 5, lines 47 - 55). 

Regarding claim 14, Tuomi teaches displaying a portion of the image of the 
scene having adjustable pixel values (i.e. primary depth information of the pixel 2102 
will change to the Z-value of the polygon; see col. 1 3, lines 4 - 36). 

Regarding claim 16, Tuomi teaches the method operation of adjusting pixel 
values corresponding to objects within any one or both of a foreground region and a 
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background region of the captured image based upon bit values of the depth mask 
(see col. 1 1 , line 48 to col. 12, line 14). 

Regarding claim 18, Tuomi clearly teaches a first logical value (i.e. 0) assigned to 
represented to the foreground objects and a second logical value (i.e. 1) assigned to 
represented the background objects (see figures 22 and 27, col. 13 lines 56-64, col. 
17, lines 12-15). 

Regarding claim 20, Chen teaches each logical element (e.g. 218) being one or 
combination of hardware and software (see col. 7, lines 19-23 and parts list on col. 
13). 

Regarding claim 21 , Chen teaches the image capture device being a video 
capture device (cameras 11). 

Regarding claims 22 and 27, Chen teaches the depth logic (218) configured to 
periodically provide a depth mask for a sequence of video frames captures by the 
video capture device (see col. 2, lines 35-49). 

Regarding claims 23 and 28, Chen clearly teaches the image capture parameter 
being brightness (i.e. intensity) (see col. 7, lines 29-34). 

Regarding claim 30, it claims the data for each pixel including distance 
information as recited in claim 1 1. Thus it is analyzed as previously discussed with 
respect to the claim 1 1 above. 
4. Claims 1-5, and 7-8 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Gvili (Depth Keying, Proceedings of SPIE-IS&T Vol. 5006, 2003, pp 564-574) in view 
of Tuomi. 



Application/Control Number: 10/758,817 Page 6 

Art Unit: 2609 

Regarding claim 1 , Gvili discloses a method for differentiating between 
foreground objects and background objects within a scene being captured through 
an image capture device comprising: 

- emitting a ray of light from a light source toward an object of the scene (i.e. 
generating a light wall, see para. 3.1 , pp 566 - 567); 

- opening and closing an aperture cover allowing access to a sensor of the 
image capture device for reflected light from the light source (i.e. deploying a 
fast image shutter in front of the CCD chip) (page 567, lines 4-5); 

- and generating a depth mask identifying objects within a foreground region of 
the scene based upon the light captured during the set time (i.e. the collected 
light by each pixel is inversely proportional to the depth of the specific pixel, 
and normalized depth of pixel can be calculated) (see pages 568-569). 

Gvili does not explicitly disclose the step of adjusting image capture device 
parameters according to bit values of the depth mask. However, Tuomi teaches a 
method of adjusting image capture device parameters according to bit values of the 
depth mask prior to capturing a subsequent corresponding image of the scene (see 
col. 13, lines 4 - 36). Therefore, it would have been obvious to one of ordinary skill in 
the art at the time the invention was made to modify Gvili by adjusting pixel values 
with relating to the depth value or a Z-value. The depth information in the Z-buffer 
2606 is primarily used during the rendering process to determine which pixel is in the 
foreground (see figures 21 and 26, and col. 16, lines 48 - 57). The motivation to do 
so would have benefit of reducing the amount of memory required for each pixel 
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stored in a frame buffer because only the color values for the antialiasing mask 
operation need to be stored, thus reducing the size of the frame buffer (see col. 10, 
lines 57-61 of Tuomi). 

Regarding claim 2, Tuomi teaches storing the depth mask in memory of the 
image capture device (i.e. provided a Z-buffer 1902 for storing the Z-values relating 
to the depth of the pixel; see col. 12, lines, 15 - 36). 

Regarding claims 3 and 8, Gvili teaches pulsing infrared light from the light 
source (i.e. IR Laser diodes) (see page 567, lines 16-17). 

Regarding claim 4, Gvili teaches the method operation of opening an aperture 
cover allowing access to a sensor of the image capture device includes, receiving 
reflected light from the objects within the foreground region, (i.e. a simple key of 
foreground objects can be generated by setting the depth measurement window to 
include the ranges, in which the foreground objects are located, see figure 4 of 
Gvili). In such setting, the camera captures light reflected from every object inside 
the depth measurement window, and ignores light reflected from objects outside the 
window (see figure 5 and para 3.2, pp 568). 

Regarding claim 5, Gvili teaches identifying objects within the foreground region 
with a first bit value (1), and identifying object within the background region with a 
second bit (0) (see page 569, lines 9-17 and page 570, lines 2-6). 

Regarding claim 7, Gvili teaches the image capture device parameters including 
focus (i.e. zoom) (see page 568, lines 1-3). 
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5. Claim 6 is rejected under 35 U.S.C. 103(a) as being unpatentable over Gvili in view 
of Tuomi as applied to claim 1 above, and further in view of Shimada (US 
2003/0025822). 

Regarding claim 6, note the discussion of Gvili and Tuomi above. Gvili and 
Tuomi do not teach the step of determining an optimal amount of light based upon 
the depth mask and adjusting the aperture cover to allow the optimal amount of light 
into the image capture device. However, Shimada teaches adjusting the aperture 
cover to allow the optimal amount of light into the image capture device (see [0047] 
and [0050]). It would have been obvious to one of ordinary skill in the art at the time 
this invention was made to have use the aperture cover for adjusting the light as 
taught by Shimada to the camera of Chen as modified by Tuomi so that the 
. brightness of the image is sufficient with the steady light rays, thus improve a quality 
of a photographed image (see [0009] and [0076] of Shimada). 

6. Claims 15, 26, and 31 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Chen in view of Tuomi as applied to claims 9 and 25 above, and further in view 
ofSawano (US 6,677,967). 

Regarding claim 15, note the discussion of Chen and Tuomi above. Chen and 
Tuomi do not teach the image of the scene being an image of a participant for use in 
an interactive game application. However, Sawano teaches video game system (see 
figure 1 ) for capturing video signals supplied from a given video source and 
processes data by using the image based on the captured video signals to create an 
image suited to user's preference (see col. 5, lines 15 - 24). Therefore, it would have 
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been obvious to one of ordinary skill in the art at the time the invention was made to 
have used the scene as game application as taught by Sawano to the scene of 
Chen as modified by Tuomi because it provides more enjoyable image creation from 
a user by combining the captured image with the original image on the screen (see 
col. 3, lines 40-44 of Sawano). 

Regarding claim 26, Sawano teaches capture device (CPU 13) including video 
game console (see figures 1-2). 

Regarding claim 31 , it claims that the scene image data includes an image of a 
person and the image of the person being incorporated into a video game for 
interaction therein as stated in claim 15. Thus it is analyzed as previously discussed 
with respect to the claim 1 5 above. 
7. Claim 19 is rejected under 35 U.S.C. 103(a) as being unpatentable over Chen in 
view of Tuomi as applied to claim 17 above, and further in view of Gvili. 

Regarding claim 19, note the discussion of Chen and Tuomi above. Both do not 
teach a sensor in communication with the depth logic, the sensor configured to 
receive a light signal reflected from one of the foreground objects. However, Gvili 
teaches the a sensor (i.e. camera) in communication with the depth logic (i.e. 
software or hardware to generate depth key setting), the sensor configured to 
receive a light signal reflected from one of the foreground objects, the receipt of the 
light signal indicating a location corresponding to one of the foreground objects (see 
page 568, 3.2 "Depth Key Setting"). It would have been obvious to one of ordinary 
skill in the art at the time the invention was made to have provided a sensor as 
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taught by Gvili to the camera of Chen as modified by Tuomi so that it can provide 
separation of the foreground object from its background objects accurately (see 
page 564, Abstract of Gvili). 
8. Claim 29 is rejected under 35 U.S.C. 103(a) as being unpatentable over Chen in 
view of Tuomi, and further in view of Kozlowski (US 2005/0105777). 

Regarding claim 29, note the discussion of Chen and Tuomi above. Both do not 
teach that image capture device is a webcam. However, Kozlowski teaches the 
image capture device is a webcam (webcam 134). It would have been obvious to 
one of ordinary skill in the art at the time the invention was made to have provided a 
webcam as taught by Kozlowski to the camera of Chen as modified by Tuomi so that 
it may be used to capture a subject video sequence of a subject person and editing it 
(see pp [0024] and [0025]). 

Conclusion 

15. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

- Mori (US 5,455,685) is cited to teach an exposure control device for a video 
camera having automatic gain control by which both stability and instantaneous 
regulation of exposure can be satisfied in accordance with a photographing 
scene. 

- Pan et al. (US 6,785,329) is cited to teach an automatic video object extraction 
that defines substantially precise objects. 
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- Long (US 7,161,634) is cited to teach adjusting the color depth of an RGB signal 
using error diffusion without the using an expensive frame buffer. 

- Swain et al. (US 6,393,142) is cited to teach a method and apparatus provide a 
technique for generating a mesh based on an image view. 

Inquiries 

1 6. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Kent Wang whose telephone number is 571-270- 
1703. The examiner can normally be reached on 7:30 A.M. - 5:00 PM (every other 
Friday off). If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, Chanh Nguyen can be reached on 571-272-7772. The fax 
phone number for the organization where this application or proceeding is assigned 
is 571-270-8300. Information regarding the status of an application may be obtained 
from the Patent Application Information Retrieval (PAIR) system. Status information 
for published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR 
only. For more information about the PAIR system, see http://pair-direct.uspto.gov. 
Should you have questions on access to the Private PAIR system, contact the 
Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like 
assistance from a USPTO Customer Service Representative or access to the 
automated information system, call 800-786-9199 (IN USA OR CANADA) or 571- 
272-1000. 
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